This invention is related to an architecture and an operation of a solid state imaging apparatus, particularly to a solid state imaging apparatus and a system for capturing a plurality of frames at an ultra-high frame rate.
T. Etoh et al., An in-situ storage image sensor of 1,000,000 fps with slanted linear CCD storage, J. of the Institute of Image Information and Television Engineers, 65(3), 483-486, 2002, in Japanese discloses a technique with which a plurality of analogue CCD memories are installed in each pixel to capture images at a high frame rate. T. Etoh, et al., Backside Illuminated Image Sensor of 16,000,000 fps, 65(3), 349-353, 2011, in Japanese discloses an improved technique applied to a backside illumination structure to significantly increase the sensitivity by avoiding wasting incident light in photoelectric conversion. An ultra-high-speed video camera with an image sensor of 165,000 pixels operating at 16,000,000 frames per second (fps) was developed with this technology.
FIG. 1 is a partial plane view of the imaging apparatus disclosed in T. Etoh et al., An in-situ storage image sensor of 1,000,000 fps with slanted linear CCD storage, J. of the Institute of Image Information and Television Engineers, 65(3), 483-486, 2002, in Japanese, and FIG. 2 is a cross-section view of the backside illumination structure disclosed in T. Etoh, et al., Backside Illuminated Image Sensor of 16,000,000 fps, 65(3), 349-353, 2011, in Japanese. Hereafter, an operation mode of the background technology is briefly explained.
A signal charge collection part 1 in FIG. 1 collects and stores signal charges, transfers the signal charges to an element of an analogue CCD memory 2, and then waits for the timing at which the next frame is captured. The time required for these steps is represented by ti. The analogue CCD memory 2 is equipped with k transfer elements. When a signal charge packet is transferred from the signal charge collection part to the analogue CCD memory, signal charge packets are transferred by one stage of the analogue CCD memory in the direction shown with arrows in the figure. After the duration ti elapses, a next signal charge packet is transferred to the analogue CCD memory, and the signal charge packets stored in the analogue CCD memory are transferred by one stage of the memory in the direction of the arrows.
By repeating the operation k times, all elements of the analogue CCD memory 2 are filled with the signal charge packets with the time interval ti. The last element of the analogue CCD memory 2 is connected to a vertical readout CCD 3 (vertical analogue memory) through a connection gate 8. Then, a combined signal transfer operation of the analogue CCD memory and the vertical readout CCD 3 starts.
The number of elements of the vertical readout CCD memory connected to one analogue CCD memory is represented by j. Then, (k+j) signal charge packets with the interval of ti are finally stored in the analogue CCD memory 2 and the vertical readout CCD 3.
Since every pixel 9 includes the signal charge collection part and the attached analogue CCD memory, the signal charges are simultaneously collected and transferred at all pixels. Since a group of image signal charge packets are recorded simultaneously in all pixels 9 at an interval of ti, the signal charge packets for (k+j) frames (signal charge packets for one frame) with the interval of ti are stored in the imaging apparatus.
In the next step, the vertical readout CCD 3 is operated to transfer the signal charge packets to a horizontal transfer CCD 6 in the direction shown with the arrows. After the horizontal transfer CCD with R elements are filled with the signal charge packets, the signal charge packets are transferred on the horizontal transfer CCD (from right to left in the figure) to an amplifier and a readout circuit 7, and are read out to the outside of the sensor. By repeating the operation, a series of image signals for (k+j) frames with the interval of ti are read out to the outside of the imaging apparatus.
In FIG. 1, a drain 4 discharges excessive charges from the signal charge collection part 1 when the charge collection part receives a large amount of charges, and a gate 5 controls the discharge of the excessive charges to the drain 4.
FIG. 2 is a cross-section view 10 of the backside illumination structure to explain a photoelectric conversion process and a signal charge collection process. The figure illustrates a simplified cross section along a horizontal line from a signal charge collection part to an analogue CCD memory. On a front side of a semiconductor chip made of Si, signal charge collection parts 11, collection control electrodes 12, analogue CCD memories 13, transfer electrodes of the CCD 14, n-type CCD channels 15, and channel separations 16 are formed. Hereafter, an electrode of the signal charge collection part is also referred to as a charge collection electrode or a collection electrode.
A high-concentration p-layer 20 is formed near the backside of the Si chip. The bulk of the chip includes a low-concentration p-layer 19 and a low-concentration n-layer 18 forming a junction, which is kept under a strong reverse-bias state. Therefore, as the carriers are withdrawn to the outside of the bulk, the bulk is substantially depleted, which creates an electric field toward the backside from the front side. Photons (electromagnetic waves) 21 incident on the backside of the Si are converted to a pair of electrons and holes. The signal electrons and the holes are attracted respectively to the front side and the backside by the field. The holes are discharged to the high-concentration p-layer at the backside and to the outside of the chip.
To prevent electrons attracted to the front side from migrating to the analogue CCD memory, a p-well 17 is formed as a p-well barrier layer. The structure enables collection of practically all electrons generated by the photoelectric conversion to the signal charge collection part 11. Time required for collection of an electron in the process is less than several nanoseconds. The time can be reduced to about one nanosecond.
The frame rate (the number of frames per second) of conventional high-speed imaging devices illustrated in FIG. 1 is the inverse of the frame interval ti. A technical issue of high-speed imaging apparatuses is how to reduce the frame interval ti.
The frame interval ti in the image capturing using the technology disclosed in the nonpatent literature 1 and 2 is expressed by the following expression (1):ti=tc+tt+tw  (1)
where tc is a collection time for a signal charge generated by incident light to reach a signal charge collection part, tt is a charge transfer time from the signal charge collection part to an adjacent memory, and tw is a waiting time from completion of capturing and storing image signals of one frame to start of the image capture of the next frame.
The minimum frame interval tm for the highest frame rate is expressed by the following expression (1)′ when tw is zero:tm=tc+tt  (1)′
In the signal charge collection process and the transfer process explained above, the time tc for signal charge generated by incident light to reach the signal charge collection part can be reduced to less than several nanoseconds. On the other hand, the transfer time tt to a CCD element or the transfer time from a CCD element to the next element is more than ten nanoseconds, and practically several tens of nanoseconds.
Therefore, the transfer time tt of the CCD limits the minimum frame interval tm of the frame interval ti. Thus, this invention newly provides a device in which the time tt is reduced or eliminated to further increase the frame rate.
Ultra-high-speed imaging is supported not only by CCD technology. Y. Tochigi et al., A Global-Shutter CMOS image sensor with Readout Speed of 1 Tpixel/s Burst and 780 Mpixel/s Continuous, ITE Technical Report, 36(18), 9-16, 2012, in Japanese discloses an ultra-high-speed image sensor using CMOS circuits and CMOS memories. The imaging device equipped with the image sensor has achieved 10,000,000 fps for spatial resolution of 100,000 pixels.
The image sensor has the following special features:    (1) One hundred analogue memories for each pixel are placed in a peripheral area of the image sensor chip,    (2) A plurality of wires to read out signals are placed on each column of the pixels; the number of the readout wires on each column is I/4, where I is the number of the pixels on the column; image signals captured in all pixels on the column are read out, transferred to the memories on the periphery of the pixel and temporarily recorded in the memories; and, then, the number of the readout operations to read out all image signals from all pixels on the column is only four times the time required to readout one image signal from one pixel,    (3) The capacity of each analogue signal memory element is increased by forming the memory capacitor with a combination of a conventional CMOS capacitor and a capacitor made with overlapped double polysilicon electrodes, and    (4) In addition, the sensor techniques includes new technologies such as a power source circuit installed in each pixel, and a device for noise cancellation.
The charge transfer time tt of this embodiment is also much longer than the charge collection time tc. Therefore, reduction or elimination of the charge transfer time tt is the crucial issue to further achieve a higher frame rate.
Insufficient incident light is a very severe constraint for ultra-high-speed imaging due to an extremely short frame interval. T. Etoh, et al., Backside Illuminated Image Sensor of 16,000,000 fps, 65(3), 349-353, 2011, in Japanese discloses a technology combining the technology disclosed in T. Etoh et al., An in-situ storage image sensor of 1,000,000 fps with slanted linear CCD storage, J. of the Institute of Image Information and Television Engineers, 65(3), 483-486, 2002, in Japanese with a backside illumination structure with almost no change of the circuits. A 100%-fill factor (a ratio of the area of photoreceptive area to the pixel area) can be achieved by this technology.
Furthermore, A 1.4 □m FSI sensor with novel light guiding structure consisting of stacked lightpipes, ITE Technical Report, 36(18), 37-40, 2012, in Japanese discloses a front-side illuminated image sensor to effectively utilize a large portion of light incident on a pixel with a technology combining an on-chip-microlens and a stacked light guide without using backside illumination. Though the image sensor was developed for a usual video camera, the technology can be applied to high-speed image sensors.